A portion of the disclosure of this patent document contains material, which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
1. Field of the Invention
The present invention relates to image analysis and management systems. The invention is directed to organization of large arrays of visual information and their effective management. The method of processing the xe2x80x9creal scenexe2x80x9d digital images (acquired from digital cameras, scanners, etc.) is introduced in order to retrieve images on the basis of their visual similarity to a query image.
2. Description of the Related Technology
Modem scientific and industrial societies face a strong demand to introduce powerful and effective tools to manage and organize visual information, which is stored in an electronic memory. Nowadays, the world produces hundreds of thousands of digital images every day and this number has a trend to increase. The cause is obviousxe2x80x94image input devices have become cheaper, smarter, easier to use, smaller, and more reliable. However, large image databases collected by museums, libraries and commercial warehouses (e. g. Corbis) have little worth without simple and effective access methods. The invention addresses the fundamental problemxe2x80x94search and retrieval in an image database, where a query itself is presented in a form of image. The common name is Query-By-Pictorial-Example (QBPE). This is easy to use, powerful and modem approach to the visual information organization and access.
Significant advances have been taking place in this field over the last decade. Nevertheless, modem methods have several flaws. They use clumsy mechanism of defining and registering a number of image features. This causes long and non-straightforward dialog with the user, who is held responsible to form his/her query based on plurality of said image features. Those systems are almost unable to accept full colored images (24 and more bit per pixel) as a query. Some researches suggest using primitive integral characteristics of image, like color histograms, color moments or extremely downsized copy of an image, for means of perceptual similarity search. This approach might be useful in applications where images are taken in fixed projections because it does not tolerate change of the projection, scale and, especially, change of objects composition within the image. Finally, the above mentioned flaws cause lack of accuracy and/or a too narrow applicability, which prevents broad use of those methods.
Most of conventional methods use either primitive integral image features, i.e., color histograms and color moments, or complicated sets of basic features. The invention approaches the problem of image indexing in a different way. When given image is to be inserted into a database, search index is to be created as described below. The image is being dissected on xe2x80x9cspotsxe2x80x9d of free form. Definition of a spot is similar to a dictionary meaning of this word. Spot is a connected set of pixels of similar color and brightness. Shape of a spot is represented as derivatives of Fourier transformation of a spot perimeter. Shape, color and relative position within an image compose a spot descriptor. Index of the entire image is a plurality of all its spot descriptors. Thus, each image in a database is associated with numerical index created as described above. Further, both terms will be used throughout the document: index when common meaning of this term is prevailing and spot descriptor when properties of index, made up by this embodiment, are being discussed. Such representation of an image has the following significant advantages:
There is a comparison procedure, defined for any two spots, to get their perceptual similarity in numerical form.
Effectiveness. Indexes comparison procedure is quick (computationally inexpensive).
Compactness. Size of the index depends on amount of information image carries but, on average, it is thousand times less that image data itself. Usually, it does not exceed 5-10 hundred bytes.
Intrinsic separation, which means that an object represented by a group of spots can be positively identified in other images regardless to the rest of their content. For example, image of a telephone receiver will be found presented standalone as well as within the image containing many other objects (FIG. 6).
Accuracy or capability to disregard subtle differences and noise while detecting common traits in image areas, which have totally different pixel representation.